Louise: A Meta-Interpretive Learner for Efficient Multi-clause Learning of Large Programs

We present Louise, a new Meta-Interpretive Learner that performs efficient multi-clause learning, implemented in Prolog. Louise is efficient enough to learn programs that are too large to be learned with the current state-of-the-art MIL system, Metagol. Louise learns by first constructing the most general program in the hypothesis space of a MIL problem and then reducing this "Top program" by Plotkin's program reduction algorithm. In this extended abstract we describe Louise's learning approach and experimentally demonstrate that Louise can learn programs that are too large to be learned by our implementation of Metagol, Thelma

Inductive Acquisition of Expert Knowledge

Expert systems divide neatly into two categories: those in which ( 1) the expert decisions result in changes to some external environment (control systems), and (2) the expert decisions merely seek to describe the environment (classification systems). Both the explanation of computer-based reasoning and the "bottleneck" (Feigenbaum, 1979) of knowledge acquisition are major issues in expert systems research. We have contributed to these areas of research in two ways. Firstly, we have implemented an expert system shell, the Mugol environment, which facilitates knowledge acquisition by inductive inference and provides automatic explanation of run-time reasoning on demand. RuleMaster, a commercial version of this environment, has been used to advantage industrially in the construction and testing of two large classification systems. Secondly, we have investigated a new technique called sequence induction which can be used in the construction of control systems. Sequence induction is based on theoretical work in grammatical learning. We have improved existing grammatical learning algorithms as well as suggesting and theoretically characterising new ones. These algorithms have been successfully applied to the acquisition of knowledge for a diverse set of control systems, including inductive construction of robot plans and chess end-game strategies

Explanatory machine learning for sequential human teaching

The topic of comprehensibility of machine-learned theories has recently drawn increasing attention. Inductive Logic Programming (ILP) uses logic programming to derive logic theories from small data based on abduction and induction techniques. Learned theories are represented in the form of rules as declarative descriptions of obtained knowledge. In earlier work, the authors provided the first evidence of a measurable increase in human comprehension based on machine-learned logic rules for simple classification tasks. In a later study, it was found that the presentation of machine-learned explanations to humans can produce both beneficial and harmful effects in the context of game learning. We continue our investigation of comprehensibility by examining the effects of the ordering of concept presentations on human comprehension. In this work, we examine the explanatory effects of curriculum order and the presence of machine-learned explanations for sequential problem-solving. We show that 1) there exist tasks A and B such that learning A before B has a better human comprehension with respect to learning B before A and 2) there exist tasks A and B such that the presence of explanations when learning A contributes to improved human comprehension when subsequently learning B. We propose a framework for the effects of sequential teaching on comprehension based on an existing definition of comprehensibility and provide evidence for support from data collected in human trials. Empirical results show that sequential teaching of concepts with increasing complexity a) has a beneficial effect on human comprehension and b) leads to human re-discovery of divide-and-conquer problem-solving strategies, and c) studying machine-learned explanations allows adaptations of human problem-solving strategy with better performance.Comment: Submitted to the International Joint Conference on Learning & Reasoning (IJCLR) 202

Inductive logic programming at 30

Inductive logic programming (ILP) is a form of logic-based machine learning. The goal of ILP is to induce a hypothesis (a logic program) that generalises given training examples and background knowledge. As ILP turns 30, we survey recent work in the field. In this survey, we focus on (i) new meta-level search methods, (ii) techniques for learning recursive programs that generalise from few examples, (iii) new approaches for predicate invention, and (iv) the use of different technologies, notably answer set programming and neural networks. We conclude by discussing some of the current limitations of ILP and discuss directions for future research.Comment: Extension of IJCAI20 survey paper. arXiv admin note: substantial text overlap with arXiv:2002.11002, arXiv:2008.0791

05051 Abstracts Collection -- Probabilistic, Logical and Relational Learning - Towards a Synthesis

From 30.01.05 to 04.02.05, the Dagstuhl Seminar 05051 ``Probabilistic, Logical and Relational Learning - Towards a Synthesis\u27\u27 was held in the International Conference and Research Center (IBFI), Schloss Dagstuhl. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations given during the seminar as well as abstracts of seminar results and ideas are put together in this paper. The first section describes the seminar topics and goals in general. Links to extended abstracts or full papers are provided, if available

Inductive programming meets the real world

© Gulwani, S. et al. | ACM 2015. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Communications of the ACM, http://dx.doi.org/10.1145/2736282[EN] Since most end users lack programming skills they often spend considerable time and effort performing tedious and repetitive tasks such as capitalizing a column of names manually. Inductive Programming has a long research tradition and recent developments demonstrate it can liberate users from many tasks of this kind.Gulwani, S.; Hernández-Orallo, J.; Kitzelmann, E.; Muggleton, SH.; Schmid, U.; Zorn, B. (2015). Inductive programming meets the real world. Communications of the ACM. 58(11):90-99. doi:10.1145/2736282S90995811Bengio, Y., Courville, A. and Vincent, P. Representation learning: A review and new perspectives.Pattern Analy. Machine Intell. 35, 8 (2013), 1798--1828.Bielawski, B. Using the convertfrom-string cmdlet to parse structured text.PowerShell Magazine, (Sept. 9, 2004); http://www.powershellmagazine.com/2014/09/09/using-the-convertfrom-string-cmdlet-to-parse-structured-text/Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka-Jr, E.R. and T.M. Mitchell, T.M. Toward an architecture for never-ending language learning. InAAAI, 2010.Chandola, V., Banerjee, A. and V. Kumar, V. Anomaly detection: A survey.ACM Computing Surveys 41, 3 (2009), 15.Cypher, A. (Ed).Watch What I Do: Programming by Demonstration.MIT Press, Cambridge, MA, 1993.Ferri-Ramírez, C., Hernández-Orallo, J. and Ramírez-Quintana, M.J. Incremental learning of functional logic programs. InProceedings of FLOPS, 2001, 233--247.Flener, P. and Schmid, U. An introduction to inductive programming.AI Review 29, 1 (2009), 45--62.Gulwani, S. Dimensions in program synthesis. InProceedings of PPDP, 2010.Gulwani, S. Automating string processing in spreadsheets using input-output examples. InProceedings of POPL, 2011; http://research.microsoft.com/users/sumitg/flashfill.html.Gulwani, S. Example-based learning in computer-aided STEM education.Commun. ACM 57, 8 (Aug 2014), 70--80.Gulwani, S., Harris, W. and Singh, R. Spreadsheet data manipulation using examples.Commun. ACM 55, 8 (Aug. 2012), 97--105.Henderson, R.J. and Muggleton, S.H. Automatic invention of functional abstractions.Latest Advances in Inductive Logic Programming, 2012.Hernández-Orallo, J. Deep knowledge: Inductive programming as an answer, Dagstuhl TR 13502, 2013.Hofmann, M. and Kitzelmann, E. I/O guided detection of list catamorphisms---towards problem specific use of program templates in IP. InACM SIGPLAN PEPM, 2010.Jha, J., Gulwani, S., Seshia, S. and Tiwari, A. Oracle-guided component-based program synthesis. InProceedings of the ICSE, 2010.Katayama, S. Efficient exhaustive generation of functional programs using Monte-Carlo search with iterative deepening. InProceedings of PRICAI, 2008.Kitzelmann, E. Analytical inductive functional programming.LOPSTR 2008, LNCS 5438.Springer, 2009, 87--102.Kitzelmann, E. Inductive programming: A survey of program synthesis techniques. InAAIP, Springer, 2010, 50--73.Kitzelmann, E. and Schmid, U. Inductive synthesis of functional programs: An explanation based generalization approach.J. Machine Learning Research 7, (Feb. 2006), 429--454.Kotovsky, K., Hayes, J.R. and Simon, H.A. Why are some problems hard? Evidence from Tower of Hanoi.Cognitive Psychology 17, 2 (1985), 248--294.Lau, T.A. Why programming-by-demonstration systems fail: Lessons learned for usable AI.AI Mag. 30, 4, (2009), 65--67.Lau, T.A., Wolfman, S.A., Domingos, P. and Weld, D.S. Programming by demonstration using version space algebra.Machine Learning 53, 1-2 (2003), 111--156.Le, V. and Gulwani, S. FlashExtract: A framework for data extraction by examples. InProceedings of PLDI, 2014.Lieberman, H. (Ed).Your Wish is My Command: Programming by Example.Morgan Kaufmann, 2001.Lin, D., Dechter, E., Ellis, K., Tenenbaum, J.B. and Muggleton, S.H. Bias reformulation for one-shot function induction. InProceedings of ECAI, 2014.Marcus, G.F. The Algebraic Mind.Integrating Connectionism and Cognitive Science.Bradford, Cambridge, MA, 2001.Martìnez-Plumed, C. Ferri, Hernández-Orallo, J. and M.J. Ramírez-Quintana. On the definition of a general learning system with user-defined operators.arXiv preprint arXiv:1311.4235, 2013.Menon, A., Tamuz, O., Gulwani, S., Lampson, B. and Kalai, A. A machine learning framework for programming by example. InProceedings of the ICML, 2013.Miller, R.C. and Myers, B.A. Multiple selections in smart text editing. InProceedings of IUI, 2002, 103--110.Muggleton, S.H. Inductive Logic Programming.New Generation Computing 8, 4 (1991), 295--318.Muggleton, S.H. and Lin, D. Meta-interpretive learning of higher-order dyadic datalog: Predicate invention revisited.IJCAI 2013, 1551--1557.Muggleton, S.H., Lin, D., Pahlavi, N. and Tamaddoni-Nezhad, A. Meta-interpretive learning: application to grammatical inference.Machine Learning 94(2014), 25--49.Muggleton, S.H., De Raedt, L., Poole, D., Bratko, I., Flach, P. and Inoue, P. ILP turns 20: Biography and future challenges.Machine Learning 86, 1 (2011), 3--23.Olsson, R. Inductive functional programming using incremental program transformation.Artificial Intelligence 74, 1 (1995), 55--83.Perelman, D., Gulwani, S., Grossman, D. and Provost, P. Test-driven synthesis.PLDI, 2014.Raza, M., Gulwani, S. and Milic-Frayling, N. Programming by example using least general generalizations.AAAI, 2014.Schmid, U. and Kitzelmann, E. Inductive rule learning on the knowledge level.Cognitive Systems Research 12, 3 (2011), 237--248.Schmid, U. and Wysotzki, F. Induction of recursive program schemes.ECML 1398 LNAI(1998), 214--225.Shapiro, E.Y. An algorithm that infers theories from facts.IJCAI(1981), 446--451.Solar-Lezama, A.Program Synthesis by Sketching.Ph.D thesis, UC Berkeley, 2008.Summers, P.D. A methodology for LISP program construction from examples.JACM 24, 1 (1977), 162--175.Tenenbaum, J.B., Griffiths, T.L. and Kemp, C. Theory-based Bayesian models of inductive learning and reasoning.Trends in Cognitive Sciences 10, 7 (2006), 309--318.Young, S. Cognitive user interfaces.IEEE Signal Processing 27, 3 (2010), 128--140

Ultra-Strong Machine Learning: comprehensibility of programs learned with ILP

During the 1980s Michie defined Machine Learning in terms of two orthogonal axes of performance: predictive accuracy and comprehensibility of generated hypotheses. Since predictive accuracy was readily measurable and comprehensibility not so, later definitions in the 1990s, such as Mitchell’s, tended to use a one-dimensional approach to Machine Learning based solely on predictive accuracy, ultimately favouring statistical over symbolic Machine Learning approaches. In this paper we provide a definition of comprehensibility of hypotheses which can be estimated using human participant trials. We present two sets of experiments testing human comprehensibility of logic programs. In the first experiment we test human comprehensibility with and without predicate invention. Results indicate comprehensibility is affected not only by the complexity of the presented program but also by the existence of anonymous predicate symbols. In the second experiment we directly test whether any state-of-the-art ILP systems are ultra-strong learners in Michie’s sense, and select the Metagol system for use in humans trials. Results show participants were not able to learn the relational concept on their own from a set of examples but they were able to apply the relational definition provided by the ILP system correctly. This implies the existence of a class of relational concepts which are hard to acquire for humans, though easy to understand given an abstract explanation. We believe improved understanding of this class could have potential relevance to contexts involving human learning, teaching and verbal interaction

Human Comprehensible Active Learning of Genome-Scale Metabolic Networks

An important application of Synthetic Biology is the engineering of the host cell system to yield useful products. However, an increase in the scale of the host system leads to huge design space and requires a large number of validation trials with high experimental costs. A comprehensible machine learning approach that efficiently explores the hypothesis space and guides experimental design is urgently needed for the Design-Build-Test-Learn (DBTL) cycle of the host cell system. We introduce a novel machine learning framework ILP-iML1515 based on Inductive Logic Programming (ILP) that performs abductive logical reasoning and actively learns from training examples. In contrast to numerical models, ILP-iML1515 is built on comprehensible logical representations of a genome-scale metabolic model and can update the model by learning new logical structures from auxotrophic mutant trials. The ILP-iML1515 framework 1) allows high-throughput simulations and 2) actively selects experiments that reduce the experimental cost of learning gene functions in comparison to randomly selected experiments.Comment: Invited presentation for AAAI Spring Symposium Series 2023 on Computational Scientific Discover